Almost all modern service providers use the internet to provide service offerings to potential consumers. The service offerings are generally provided in the form of software applications which operate using dedicated resources of the service provider. Together, the applications and the resources dedicated to the applications form a system within the service provider infrastructure that supports the service offering.
To maintain goodwill, it is desirable that any system that is released by a service provider be free from defects which may create delays, corrupt customer data or otherwise adversely affect system performance. System problems, such as those caused by poor application performance or resource deficiencies may reduce customer confidence, degrade service provider goodwill and potentially result in loss of sales. Accordingly, before any system is released, it is important to test the operation of the system in the environment in which it ultimately will be utilized.
Performance test tools are commercially available that can be used to identify issues with software and hardware prior to the general release. Performance test tools simulate real-time workloads to “exercise” the system under test under virtual real-world conditions. Performance test tools can include software running on the system under test, software running on a remote computer that manages computer tests to interact with the system under test, or both. Performance test may be used to determine how the system under test will perform under a particular collection of workloads. Workload simulation forms an integral part of performance test. In workload simulation, different workloads are generated and subjected upon a system under test, in order to simulate the conditions under which the computing system will operate.
One challenge that is faced by performance tests is the increasing diversity and complexity of the network environment used by consumers. Systems must be able to interact both with consumers and with systems hosted by other service providers, over a variety of geographically diverse networks, using computers operating under a variety of different operating systems. It is difficult for existing performance test tools to keep pace with the dynamic network environment in which a system may be deployed.